Indirect measures of self-esteem 1 Running Head: INDIRECT MEASURES OF SELF-ESTEEM Through a Glass, Less Darkly? Reassessing Convergent and Discriminant Validity in Measures of Implicit Self-Esteem

Self-esteem has been traditionally assessed via self-report (explicit self-esteem: ESE). However, the limitations of self-report have prompted efforts to assess self-esteem indirectly (implicit self-esteem: ISE). It has been theorized that ISE and ESE reflect the operation of largely distinct mental systems. However, although low correlations between measures of ISE and ESE empirically support their discriminant validity, similarly low correlations between different measures of ISE do not support their convergent validity. We explored whether such patterns would re-emerge if more newly developed, specific, and reliable ISE measures were used. They did, although some convergent validity among ISE measures emerged once confounds due to conceptual mismatch, individual differences, and random variability were minimized. Nonetheless, low correlations among ISE measures are not primarily due to the usual psychometric suspects, and may be the result of other factors including subtle differences between structural features of such measures. Indirect measures of self-esteem 3 Through a Glass, Less Darkly? Reassessing Convergent and Discriminant Validity in Measures of Implicit Self-Esteem To investigate people’s attitude towards themselves—their self-esteem— psychologists have traditionally relied on self-report (explicit self-esteem or ESE; Rosenberg, 1965). Fortunately, when reporting their self-esteem, people are reasonably knowledgeable about themselves, honest with themselves, and honest with others. Nonetheless, people sometimes lack self-insight (“How do I feel about myself really?”; Wilson, 2002), deceive others (“I really think I’m useless, but I better pretend to be great!”; Schlenker & Leary, 1982), or even deceive themselves (“I’m great—even if everyone hates me!”; Paulhus, Fridhandler, & Hayes, 1997). Hence, self-reports of self-esteem, though tolerably valid, still contain some systematic error. One possible way to curtail such error is to employ indirect measures of self-esteem (Greenwald & Farnham, 2000). Such measures are designed to reveal people’s attitude towards themselves from their reactions to self-related stimuli (e.g., first and last names, firstpersonal pronouns), typically under conditions where people are either unaware of, or lack control over, the measurement process (e.g., Rudolph, Schröder, & Schütz, 2006). Consider unawareness: the initials preference task (IPT; Koole, Dijksterhuis, & van Knippenberg, 2001) requires respondents to rate all letters of the alphabet for likeability, whereupon people typically exhibit an unknowing preference for their initials. 1 Or consider uncontrollability: the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) requires respondents to co-classify self-related and self-unrelated stimuli alongside positive and negative stimuli. Respondents are also required to go as quickly as they can without making errors. However, they typically find responding more difficult—and hence go more slowly—when the four target categories are configured one way (e.g., Self with Bad, Non-Self with Good) rather than another (e.g., Self with Good, Non-Self with Bad). What indirect measures assess is often Indirect measures of self-esteem 4 termed implicit self-esteem (ISE). The properties of ISE have been assumed to reflect properties of the indirect measures used to assess it (e.g., ISE is unconscious and automatic; Greenwald & Farnham, 2000). However, it remains controversial whether and to what extent indirect measures operate via wholly implicit processes (for a discussion, see De Houwer & Moors, 2007). Standard dual-process models (e.g., Strack & Deutsch, 2004) suggest that ISE and ESE, being subserved by modular cognitive systems, should yield measures that are largely independent. In addition, different measures of each construct, in virtue of tapping into the same modular system, should exhibit reasonable intercorrelations (though see Marsh & Craven, 2006; Sakellaropoulo & Baldwin, 2007). Given such expectations, prior empirical research has yielded two patterns, one reassuring, the other troubling. The first suggests discriminant validity. In particular, measures of ISE and ESE typically show weak correlations (Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005), although these can be augmented under theoretically specified conditions (e.g., Jordan, Whitfield, & ZeiglerHill, 2007; Koole et al., 2001; Olson, Fazio, & Hermann, 2007). However, the second pattern suggests a lack of convergent validity: different measures of ISE typically fail to exhibit the predicted intercorrelations (Bosson, Swann, & Pennebaker, 2000). If valid, this pattern implies one of two things: either (a) ISE exists and is heterogeneous; or (b) ISE does not even exist. The matter remains unresolved. Nonetheless, most measures of ISE do converge insofar as they register a pronounced average self-positivity bias (e.g., Greenwald & Farnham, 2000; Gregg & Sedikides, 2008). In addition, such measures exhibit meaningful antecedents and consequences (see Koole & DeHart, 2007, for an overview). Hence, there are some reasons to believe (a) over (b). However, an even gloomier possibility exists: both patterns could be artifacts of measurement error. Measures of ISE have a reputation for unreliability (Bosson et al., 2000). Indirect measures of self-esteem 5 Such unreliability could obscure latent correlations, and falsely suggest that ESE and ISE diverge when they do not, or that different indices of ISE fail to converge when they do. In this article, we reconsider the convergent and discriminant validity of ISE and ESE. In particular, we investigate whether and to what extent two factors—(a) the reliability and sensitivity to self-positivity bias of different measures of ISE, and (b) the conceptual correspondence between what different measures of ISE assess—moderate the relation between measures of ISE and ESE, and between different measures of ISE. We then attempt to formulate concrete and constructive recommendations for future research, and make some empirically informed theoretical interpretations. Some years ago, a study concluded that the IAT and the IPT were the most reliable and valid measures of ISE available (Bosson et al., 2000). Since then, however, indirect measures have proliferated. In particular, three new measures have emerged that—unlike the IAT—permit associations toward an object to be assessed in isolation: the Single-Category IAT (SC-IAT; Karpinski & Steinman, 2006), the Extrinsic Affective Simon Task (EAST; De Houwer, 2003), and the Go/No-Go Association Task (GNAT; Nosek & Banaji, 2001). These new measures are of interest, because they assess self-related evaluations independently of other-related evaluations—an advantage, given that variations in the theoretically irrelevant non-self category confound performance on the self-esteem IAT (Karpinski, 2004). In addition, due to acknowledged problems concerning the effect size and reliability of the EAST, an improved variant of the EAST, namely the Identification EAST (ID-EAST), has been devised (De Houwer & De Bruycker, 2007). Furthermore, the present authors have devised a potentially more reliable version of the IPT, namely the Duplicate IPT (D-IPT). To update the literature, we conducted three studies to compare and contrast the older IAT and IPT with the newer SC-IAT, EAST, ID-EAST, GNAT, and D-IPT as putative indices of ISE. In addition to using new measures of ISE, we also applied more recently developed Indirect measures of self-esteem 6 algorithms (e.g., D-index; Greenwald, Nosek, & Banaji, 2003) to maximize validity, and employed standard indices of internal consistency (based on equivalent split-halves, and incorporating warranted Spearman-Brown adjustments). STUDY 1 In our first study, we evaluated three different measures of ISE: an IPT, an EAST, and an IAT. We quantified their internal consistency, their test-retest stability one-week apart, their intercorrelations, and their correlations with ESE.